SEPARATION CUTOFFS FOR RANDOM WALK ON 
IRREDUCIBLE REPRESENTATIONS 



JASON FULMAN 

Abstract. Random walk on the irreducible representations of the sym- 
metric and general linear groups is studied. A separation distance cutoff 
is proved and the exact separation distance asymptotics are determined. 
A key tool is a method for writing the multiplicities in the Kronecker ten- 
sor powers of a fixed representation as a sum of non-negative terms. Con- 
nections are made with the Lagrange-Sylvester interpolation approach 
to Markov chains. 



1. Introduction 

The study of convergence rates of random walk on a finite group is a rich 
subject; three excellent surveys are [3d], [Sal] and |Dlj . In recent papers 
([F1],[F3],[F6]), the author posed and studied a dual question, namely what 
can be said about the convergence rate of random walk on Jrr(G), the set 
of irreducible representations of a finite group G. 

To define the random walk on Irr(G), let r\ be a (not necessarily irre- 
ducible) representation of G whose character is real valued. From an irre- 
ducible representation A, one transitions to the irreducible representation p 
with probability 

;= d,m,(A®^ 
d\d v 

Here d\ denotes the dimension of A and m p {\ ® rf) denotes the multiplicity 
of p in the tensor product (also called the Kronecker product) of A and 77. 
Whereas random walk on G has the uniform measure as a stationary distri- 
bution, random walk on Irr(G) has the Plancherel measure as a stationary 
distribution. The Plancherel measure assigns a representation A probability 

\G\- 

There are many motivations for the study of these random walks: six 
motivations (with literature references and discussion) appear in the intro- 
duction of the recent paper [F6j . There is no need to repeat the discussion 
here, but let us just mention that similar processes have been studied for 
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compact Lie groups and Lie algebras ([ER|,[BBOJ), and that the decom- 
position of iterated tensor products of finite groups has been studied by 
combinatorialists ( |F3j . [GC] . [GKj ) . Moreover, these random walks arise in 
quantum computing ([MRJ,[F6]) and have been used to derive the first er- 
ror bounds in limit theorems for the distribution of random character ratios 
([FT], [F4j ) . As is clear from |F6J and this paper, these random walks are 
also a tractable testing ground for results in Markov chain theory. 

To illustrate a result in this paper, we discuss the case of random walk 
on Irr(S n ). To do this, recall that two commonly used distances between 
probability distributions P, Q on a finite set X are total variation distance 



IP-QW-^FW-QW 



2 



and separation distance 



s(P, Q) := max 



1 P(x) 



Q{x) 



The following recent result gave a sharp total variation distance convergence 
rate estimate. 

Theorem 1.1. (^F6]J Let G be the symmetric group S n and let ir be the 

Plancherel measure of G. Let r\ be the n- dimensional defining representation 
of S n . Let K r denote the distribution of random walk on Irr{G) after r 
steps, started from the trivial representation. 

(1) If r = ^nlog(n) + cn with c > 1 then 

-2c 

HiT-Trll < - — . 



(2) If r = ^nlog(n) — cn with < c < | log(n), then there is a universal 
constant a ( independent of c,n) so that 

||i^ r -vr|| > l-ae" 4c . 

This paper gives precise separation distance asymptotics. Letting s(r) 
denote the separation distance after r steps, it will be shown that for the 
walk in Theorem 11.11 and c fixed in R, 



a(nlog(n) + cn) = 1 - e~ e "(1 + e" c ) + O 1 l ° g ^ 



n 



This expression goes to as c — > oo and to 1 as c — > — oo and a cutoff 
(defined precisely in Section [2]) occurs since cn = o(n log(n)). Note that 
whereas the total variation cutoff occurs at time inlog(n), the separation 
distance cutoff occurs at time nlog(n). The proof of the separation distance 
asymptotics (and also the corresponding result for GL(n, q)) consists of two 
steps: 



SEPARATION CUTOFFS FOR RANDOM WALK 



3 



(1) One must determine at which element A of Irr{G) the separation 
distance is attained. This is equivalent to finding the A which mini- 
mizes mx }^ - • This is tricky since the usual formula for multiplicities 
in tensor products involves character values and so both positive 
and negative terms. Our solution is to give a subtle rewriting of this 
expression as a sum of non- negative terms. 

(2) Once one knows at which representation the separation distance is 
attained, one needs a formula for the separation distance. For the 
cases in this paper we indicate how to do this using combinatorial ar- 
guments and the diagonalization (i.e. eigenvalues and eigenvectors) 
of the random walk on Irr{G). 

A rather remarkable fact is that for the cases studied in this paper, one can 
use Lagrange-Sylvester interpolation to carry out Step 2 knowing only the 
eigenvalues (and not the eigenvectors) of random walk on Irr(G). A similar 
trick had been usefully applied in the one-dimensional setting of birth-death 
chains ([Br|. [DSa| ). but the state spaces Irr{S n ) and Irr(GL(n,q)) are 
high-dimensional so it is interesting that the trick can be extended to this 
context. A sequel will treat combinatorial examples where similar ideas can 
be applied. 

The organization of this paper is as follows. Section [2] gives background 
from Markov chain theory and recalls the diagonalization of the Markov 
chain on Irr(G). Section [3] derives separation distance asymptotics for ran- 
dom walk on Irr{S n ) when rj is the defining representation (whose character 
on a permutation is the number of fixed points). Section 0] obtains separa- 
tion distance asymptotics for random walk on Irr(GL(n,q)) when rj is the 
representation whose character is q d ^ 9 ' , where d(g) is the dimension of the 
fixed space of g. Section [5] discusses Lagrange-Sylvester interpolation, giving 
eigenvector-free proofs of some results of Sections [3] and [U 

2. Preliminaries 

This section collects some background on finite Markov chains, using ran- 
dom walk on Irr(G) as a running example. Let X be a finite set and K a ma- 
trix indexed by X x X whose rows sum to 1. Let tt be a distribution such that 
K is reversible with respect to ir; this means that n(x)K(x, y) = ir(y)K(y, x) 
for all x, y and implies that tt is a stationary distribution for the Markov 
chain corresponding to K. 

As an example, the Markov chain on Irr(G) defined in the introduction 
is reversible with respect to the Plancherel measure tt. To see this let x 
denote the character of a representation, and recall the formula 

m p {\ ® rf) = — x x (g)x n {g)x p {g)- 

1 1 geG 

The equation ir(\)K(\, p) = n(p)K(p, A) follows because r) was assumed to 
be real valued; in fact this was the reason for imposing this condition on rj. 
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Define (/,<?) = Ylxex /( 2; )5 , ( :E ) 7r ( a; ) f° r rea l valued functions /, g on X, 
and let L 2 (ir) denote the space of such functions. Then when K is considered 
as an operator on L 2 (tt) by 

Kf(x) :=Y,K(x,y)f(y), 

y 

it is self adjoint. Hence K has an orthonormal basis of eigenvectors fi(x) 
with Kfi(x) = Pifi(x), where both fi and are real. It is easily shown 
that the eigenvalues satisfy — 1 < P\x\-l < " " " < A < A) = 1- One calls 
ergodic if \(3\ x \-i\, \Pi\ < 1- 

As an example, Lemma 12 . 1 1 determines an orthonormal basis of eigenvec- 
tors for the Markov chains on Jrr(G). 

Lemma 2.1. ( , [F2| . Proposition 2.3) Let K be the Markov chain on Irr(G) 
defined using a representation n whose character is real valued. The eigen- 
values of K are indexed by conjugacy classes C of G: 

(1) The eigenvalue parameterized by C is x . 

(2) An orthonormal basis of eigenf unctions fc in L 2 (tt) is defined by 

For instance when G = S n and r] is the n-dimensional defining represen- 
tation, the eigenvalues are ^ where 0<i<n — 2 or i = n, with multiplicity 
equal to the number of conjugacy classes of permutations with i fixed points. 
Similarly, suppose that G = GL(n, q) and that r/ is the representation whose 
character is the number of fixed vectors of g in its natural action on the n- 
dimensional vector space V. Then the eigenvalues are q~ l for i = 0, • • ■ ,n, 
with multiplicity equal to the number of conjugacy classes of elements of 
GL(n, q) with an n — i dimensional fixed space. 

A common way to quantify convergence rates of Markov chains is using 
total variation distance. Given probabilities P,Q on X, one defines the total 
variation distance between them as \\P — Q\\ = \ X^ex l^ > ( x ) ~ Q( x )\- It i s 
not hard to see that 

\\P-Q\\ = max\P(A)-Q(A)\. 

Let K r x be the probability measure given by taking r steps from the starting 
state x. Researchers in Markov chains are interested in the behavior of 

Lemma [2.21 relates total variation distance to the spectrum of K. Part 1 is 
the usual method for computing the power of a diagonalizable matrix. Part 
2 is proved in [DH] and upper bounds \ \K X — vr|| in terms of eigenvalues and 
eigenvectors and is effective in many examples; it was crucial in the proof of 
Theorem 1 1 . 1 1 stated in the introduction. 

Lemma 2.2. (1) K r (x,y) = Eifo" 1 01 fi{x) fi(y)ir{y) foranyx,y G X. 
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(2) 



y 



7r(y) 



\x\-i 

i=l 
0. 



Note that the final sum does not include i 

Another frequently used method to quantify convergence rates of Markov 
chains is to use separation distance, introduced in [ADl],[AD2j. The sepa- 
ration distance between probabilities P, Q on X is defined as 

P(x)~ 

W). 

Since \\P — Q|| = J2 x -.Q(x)>P(x) iQ( x ) ~ P( x )]i ^ is straightforward that 
\\P — Q\\ < s(P,Q). Specializing to random walk on Irr(G) started at the 
trivial representation 1, one has that 



s(P, Q) = max 



1 



7T 



max 
A 



1 



\G\K r (l, A) 



d 2 
a \ 



:m\(rj r ). Thus the separation 



Lemma 3.2 of [F6] gives that K r (l, A) 

distance is attained at the A which minimizes m j ' . 

Finally, let us give a precise definition of the cutoff phenomenon. A nice 
survey of the subject is [D2| : we use the definition from [Salj . Consider a 
family of finite sets X n , each equipped with a stationary distribution ir n , 
and with another probability measure p n that induces a random walk on 
X n . One says that there is a total variation cutoff for the family (X n , 7r n ) if 
there exists a sequence (t n ) of positive reals such that 

(1) lim^oot^ = oo; 

(2) For any e G (0, 1) and r n = [(1 + e)t n ], lim n _»oo \\p r n n - ir n \\ = 0; 

(3) For any e G (0, 1) and r n = [(1 - e)t n ], lim^oo \\p r n " - 7r n || = 1. 

For the definition of a separation cutoff, one replaces | \p r ™ — 7r n | | by s(p r ^ ,Tr n ). 



3. The symmetric group 

This section studies the random walk K on Irr(S n ) defined from the 
representation rj whose character is the number of fixed points. Although 
not needed for the results in this section, it should be mentioned that when 
Irr(S n ) is viewed as the partitions of n, the random walk K has a description 
in terms of removing and then reattaching a corner box at each step (see 
[F6J for a proof). 

The primary purpose of this section is to determine the asymptotic be- 
havior of the separation distance 



sir) = max 
A 



1 



K r {l\) 
vr(A) 
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The first step in studying s(r) is determine for which A the maximum is 
attained. Part 1 of Lemma 12.21 and Lemma 12 . 1 1 imply that 



ir(A) ^— ' V n J ^-^ d\ 

^ ' i=0 V 7 geS n :fp(g)=i 

where fp(g) is the number of fixed points of g. However since characters 
can take both positive and negative values, it is not clear which A minimizes 
this expression. 

Theorem 13.31 will circumvent this difficulty by giving an expression for 
K Jjx^ as a sum °f non- negative terms. This result was first derived in 



our earlier paper [F5J using the Robinson-Schensted-Knuth correspondence. 
The proof presented here is different and uses instead inclusion-exclusion 
and the branching rules for the irreducible representations of S n . As will be 
seen in Section [H it generalizes perfectly to the group GL(n,q). 

As a first step, the following lemma is useful. In its statement, and 
throughout this section, we assume familiarity with the concept of standard 
tableaux as in Chapters 2 and 3 of [Sag|. 

Lemma 3.1. Let d\/ n denote the number of standard tableaux of shape A//i. 
Then 

Ex, , n\ v-^ (— IV , 
* fo) = TTZ^— — d A/(n-i-j)- 

ges n -.fp(g)=i ' j=o 

Proof. Let Fix(g) denote the set of fixed points of a permutation g. Then 

E x A ^) 

geS n :fp(g)=i 

E x\g) 

g:Fix(g)={n—i+l,--- ,n} 
n—i 

M E(-!) j E E x\g) 

j=0 AC{l,-,n-i} g:Fix(g)DAU{n-i+l,--- ,n} 

n \ v-^ / I n — i 



B-^r;' E * A <»> 

j=0 g:Fix(g)D{n-i-j+l,--- ,n} 

n-i / _ \ 

E(-i) j ( n /J (« - * - [x A ], i) 



- ~7\ JT~ d X/(n-i-j)- 

The first and third equalities are since character values are invariant under 
conjugacy. The second equality is the inclusion-exclusion principle (Chapter 
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10 of [ V Wj ) . In the fourth equality, Res s n . \x X ] denotes the restriction of 

X A from S n to S n —i—j- The final equality follows from the branching rules 
for irreducible representations of symmetric groups Sag|. Note also that 
when i = 0, the set {n — i + 1, • • ■ , n} should be interpreted as the empty 
set. □ 



In what follows P(a, r, n) will denote the probability that when r balls 
are dropped at random into n boxes, there are exactly a occupied boxes. 
Lemma 13.21 gives an explicit expression for P(a,r,n). This expression is an 
exercise on page 103 of [Fe| . but since the proof is simple and motivates an 
analogous result in Section [H we include it. 

Lemma 3.2. rfFe\ ) 

*-»^(:)eh.'-->(; 1 )(^)'. 

v 7 b=n-a V 7 V 7 

Proof. Clearly P(a, r, n) is (") multiplied by the probability that the occu- 
pied boxes are the first a boxes. By the principle of inclusion and exclusion, 
this is 



a 

n' 



7 s=0 



where P<(s) is the probability that the set of occupied boxes is contained 
in {1, • • • , s}. Noting that P<(s) = (-) r , the result follows from the change 
of variables b = n — s. □ 

Theorem 13.31 gives the needed expression for ~Ry^ as a sum of non- 
negative quantities. 

Theorem 3.3. Let d\i^ denote the number of standard tableaux of shape 
A//x. Then 



— t^— = 2^ p ( a > r > n )( n - a ) 



, rf A/(n-a) 



-(A) " ' * 

Proof. As noted earlier, part 1 of Lemma 12.21 and Lemma |2 . 1 1 imply that 
2T(i,A)_£/iy £ ^ 



7r(A) ^— ' \ n J , d\ 

^ 1 i=0 V 7 geS n :fp(g)=i 



By Lemma 13. II this is 



i=o v 7 j=o J A 
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Letting a = i + j, this becomes 

r Tl\ 1 / i \ '' d x /( n -a) 

dx 

\/(n—a) 



i=0 a=i 

n! 1 / i \ a 



i\ (a — i)\ \n J d\ 
a=0 i=o v j \ / * 



2^ZJ< ' i\(a-i)\\n 

8=0 a=i 
n a 

= ££(-D a 

a=0 i=0 

Letting b = n — i, this becomes 

e(:) e (-^(,: t ) H)V.>^. 

The result follows from Lemma 13.21 □ 

Corollary 3.4. T/ie quantity n ^ is minimized for X = (l n ), correspond- 
ing to the sign representation. 

Proof. By Theorem 13.31 one wants to find A minimizing 

Note that the a = n — 1 and a = n terms in this expression are independent 
of A. Moreover, all other terms are non-negative, and vanish when A is the 
sign representation (which corresponds to the partition all of whose parts 
have size 1). The result follows. □ 

Next we use Theorem 13. 31 and Corollary 13.41 to derive both a formula and 
a precise asymptotic expression for the separation distance of the Markov 
chain K. 

Theorem 3.5. Let s{r) be the separation distance between K r started at 
the trivial representation and the Plancherel measure n. 

(1) 

n-2 / x / ■ \ r 



(2) For c fixed in R and n — > oo, 

sfnlogfn) + cn) = 1 - e~ e ~\\ + e" c ) + O ( 

\ n 

Proof. (First proof) Theorem 13.31 and Corollary 13.41 imply that 

n i 

s(r) = l-yP(a,r,n)(n-ay. iln ] /{n - a) 
t=1> V) 
= 1 — P(n, r, n) — P(n — 1, r, n). 
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By Lemma 13.21 this is equal to 



b=0 

n 



b 1 \ n J ^— ' V n — bj \ n 



6=1 



Letting i = n — b proves the first assertion. 

For the second assertion, we use asymptotics of the coupon collector's 
problem: it follows from Section 6 of |CDM] that when nlog(n) + cn balls 
are dropped into n boxes, the number of unoccupied boxes converges to a 
Poisson distribution with mean e _c , and that the error term in total variation 
distance is 0( lo ^[ )■ The chance that a Poisson random variable with mean 
e~ c takes value not equal to or 1 is 1 — e~ e c (l + e _c ), which completes 
the proof. □ 

There is a second proof of Theorem 13.51 which uses a connection with the 
top to random shuffle of the symmetric group. We prefer the first proof as 
the ideas are more elementary (one doesn't need the RSK correspondence) 
and generalize to GL(n,q) (see Section [3]). 

Proof. (Second proof) By Corollary 13.41 s(r) = 1 — n\K r (l, (l n )). Theorem 
3.1 of [F3| gives that for any shape A, one has that K r (l, A) is equal to the 
chance of obtaining a permutation with Robinson-Schensted-Knuth (RSK) 
shape A after r top to random shuffles started from the identity. The only 
permutation with RSK shape (l n ) is the "longest" permutation ir, defined 
by ir(i) = n — i + 1 for all i. It follows from Corollary 2.1 of [DFP] that 
the separation distance for the top to random shuffle is attained at this n. 
Thus the chain K and the top to random shuffle have the same separation 
distance s(r), so the result follows from page 142 of |DFP] . □ 

From the second proof the reader might think that the theory of the 
chain K can be entirely understood in terms of the top to random shuffle. 
This is untrue. For example if one measures convergence to the stationary 
distribution using total variation distance, the top to random shuffle takes 
nlog(n) + cn steps to be close to random [ADlj . but the chain K requires 
only inlog(n) + cn steps [F6| . 

As a final result, we use Corollarv l3.4l and the cycle index of the symmetric 
group to give a third proof of part 1 of Theorem 13.51 

Proof. By Corollary 13.41 s{r) = 1 — — - ■ Part 1 of Lemma 12.21 and 
Lemma 12.11 imply that 

jr(t,A) ^/«Y x\a) 



7r(A) ^— ' d\ 

y ' *=0 V 7 geS n :fp(g)=i 
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Specializing to A = (l n ) implies that 



n-l 



i=0 



( r ) = -£(-) S si 9<9)- 



g£S n :fp(g)=i 



Here sign(g) = (—l) n ~~ c ( 9 \ where c(g) is the number of cycles of g. 

A classic result in combinatorics (see |W| ) is the "cycle index" of the 
symmetric group, which states that 



„ n i(ff) 



X A 

n>l '" g£S„ j>l \ 111 I 



exp ]T 



m 



Here nj(g) is the number of cycles of length j of g. Making the substitutions 
x\ = —x, Xi = — 1 for i > 2 and replacing u by —u, the cycle index implies 
that 



1 + Yl ^7 Yl si 9 n (9) 



Jp(g) 



n>l g&Sn 



exp xu 



E 



(-»)* 



771 



m>2 

exp(xu — u) exp (log (1 + u)) 
e xu (l + u) 



Taking the coefficient of on both sides shows that if0<i<n — 1, then 



Y, si 9n(g) 

g£S n :fp(g)=i 



nl 



(-1) 



(-1) 



a — i I i— 1 

h -r 

(n — i)\ (n — i — 1)1 

in — i — 1). 



n—i+l 



7? 



The result now follows from the previous paragraph. 



□ 



4. The general linear group 

This section studies random walk on Irr(GL(n,q)) in the case that rj is 
the representation of GL(n,q) whose character is q d ^ 9 \ where d(g) is the 
dimension of the fixed space of g. As in Section [3l we aim to determine the 
asymptotic behavior of the separation distance 



s(r) = max 
A 



1 



K r (l,A) 



The first step is to find the irreducible representation A for which the 
maximum is attained. Part 1 of Lemma 12.21 and Lemma 12.11 imply that 



tt(A) 



i=0 



-r(n—i) 



E 

g€GL(n,q):d(g)= 



x A (g) 

d A 
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Since characters can take both positive and negative values, it is not at all 
clear which A minimizes this expression. As in the symmetric group case, 

the key is to find a way to write as a sum of non- negative terms. 

To begin we recall some facts about the representation theory of GL(n, q). 
A full treatment of the subject with proofs appears in [Ma], [Z]. As usual 
a partition A = (Ai, • • • , A m ) is identified with its geometric image {(i, j) : 
1 < i < m, 1 < j < Ai} and |A| = Ai + • • • + A m is the total number of boxes. 
Let Y denote the set of all partitions, including the empty partition of size 
0. 

Given an integer 1 < k < n and two characters XI1X2 of the groups 
GL(k, q) and GL(n — k,q), their parabolic induction xi X2 is the character 
of GL(n, q) induced from the parabolic subgroup of elements of the form 

P= {( g 2 ) ■9i^GL(k,q),g 2 GGL(n-k,q) 

by the function Xi(ffi)X2(fl r 2)- 

A character is called cuspidal if it is not a component of any parabolic 
induction. Let C m denote the set of cuspidal characters of GL(m, q) and 
let C = \J m >i c m; it is known that \C m \ = ± Yld\m ^(d)(q m/d - 1) where fx 
is the Moebius function. The unit character of GL(1, q) plays an important 
role and will be denoted e; it is one of the q — 1 elements of C\. Given 
a family A : C h ¥ with finitely many non-empty partitions A(c), its 
degree ||A|| is defined as Ylm>\ Scec* m m ' l^( c )l- A fundamental result is 
that the irreducible representations of GL(n, q) are in bijection with the 
families of partitions of degree n, so we also let A denote the corresponding 
representation. 

Let 

e ij ■ ■ ■ j e n be the standard basis of the vector space V on which 
GL(n,q) acts (so the kth component of ej is Sjk). Define H(k,q) as the 
subgroup of GL(n,q) consisting of g which fix all of el, • • • , ej~. Equiva- 
lently, the elements of H (k, q) are block matrices of the form 

h X 

where Ik is a k by k identity matrix, X is any k by n — k matrix with 
entries in ¥ q , and Y is any element of GL{n — k,q). Thus \H(k,q)\ = 
q k ( n ~ k ^\GL(n — k,q)\. For A an element of Irr(GL(n,q)), it will be helpful 
to let 

cfc(A)= Y, X A ^' 

geH(k,q) 

Then Cfc(A) is non-negative, since it is the product of \H(k, q) \ and the multi- 
plicity of the trivial representation of H(k, q) in the restricted representation 
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It will also be convenient to let [?1 denote the q-binomial coefficient 

( g fc _ i) . . ■(q-i)(g n - k jpCT i which is equal to the number of k dimensional 
subspaces of an n dimensional vector space over a finite field ¥ q . 

Lemma 4.1. Let A be an irreducible representation of GL(n,q). Then 



E * 

geGL(n,q) 
d(g)=i 



~ [I] 



n—i 

E 

3=0 



n — i 
3 



(-l)^)c^(A). 



Proof. Let Fix(g) denote the fixed space of g. Also if A is a set of vectors, 
(A) will denote their span. Then 



E * A (s) 



g&GL(n,q) 



E x A (5) 



g£GL(n,q) 



n— « 

["]E 



E 

j=0 HQ <el ,- ,«}> 



(-1)^) Y, x\g) 



9 eGL(n,g) 
Fix( 9 )DVK 



[I] 
["} 



E 

i=o 

n—i 

E 

3=0 



n — i 
3 

n — i 
3 



(-i)V 



E 



g£GL{n,q) 



-l)^(a)ci +i (A). 



The first and third equalities used the fact that if W\ , Wi are subspaces 
of V of equal dimension, then {g : Fix(g) = W{\ and {g : Fix(g) = W2} 
are conjugate in GL(n,q). The second equality used Moebius inversion on 
the lattice of subspaces of a vector space (Chapter 25 of the text [VW| ). 
The third equality also used the fact that the number of i + j dimensional 

subspaces of V containing (el, ■ ■ ■ , ej) is 



n — 1 
j 



□ 



In what follows we let P q (a,r,n) denote the probability that the span 
of v{, ■ ■ ■ , if r is a dimensional, where the r vectors are chosen uniformly at 
random from an n-dimensional vector space over ¥ q . Lemma 14,21 gives a 
formula for P q (a,r,n). 



Lemma 4.2. 

P q {a,r, n) 



n~\ 
a\ 



j2 (-i)M«-a)/- ( r a) ) 



b=n—a 



a 

n — b 



q 



-rb 



Proof. Clearly P q (a,r,n) is [™] multiplied by the chance that the span of 
v{ , ■ ■ ■ , v* r is exactly the a dimensional subspace consisting of vectors whose 
last n — a coordinates are 0. One applies Moebius inversion on the lattice 
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of subspaces of an n-dimensional vector space over ¥ q (Chapter 25 of [VWj 
to conclude that 



<(W)- 



=0 WC{ei,- ,ea) 



Here €{,■•■ , e n is the standard basis of V and P<(W) is the probability that 
the span of v{, ■■■ ,v r is contained in W. Clearly P<(W) = q- r i n ~ dm (W)) _ 
Thus 

p g ( ai r in ) = [ n ]f[ fl ](-ir 9 (T)r (n - s) , 

and the result follows by the change of variables b = n — s. □ 



if (i,A) 

tt(A) 



Theorem 14.31 is a key result of this section; it expresses — r-r— as a sum 
of non-negative terms. 

Theorem 4.3. 



K r (l,A) ^ 
— j-- = ^P q (a,r,n) 



Ca(A) 



Proof. Part 1 of Lemma 12.21 and Lemma 12.11 imply that 

JT(i,A) A X A (9) 



i=0 



tt(A) 

By Lemma |4.1|, this is 

£ „-,(„-„ 

i=0 j=0 

Letting a = i + j, this becomes 

i=0 a=i 
n 



E 



geGL{n,q):d(g)=i 



d\ 



n — i 
3 



«A 



n — i 
a — i 



a=0 i=0 

Setting b = n — i this becomes 
c«(A) 







— i 




a 


— i 



_ ir -^( T )^ 



/ a — i\ 
a - l „{ 2 J 



a=0 

n 



a — (n — b) 



(_l)a-(n-6) g ( a - ( r 6) ) 



= v^wn y ( _i)6-(n-a )g e-(r a) ) 

a=U b=n—a 

The result now follows from Lemma 14.21 



a 

n — b 



-rb 



□ 



Corollary 4.4. Suppose that GL(n,q) / GL(1,2). 
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(1) The quantity K Jjj^ is minimized for any A which satisfies A(e) = 0, 
and such A exist. 

(2) The separation distance s(r) of K started at the trivial representation 
is 

n 

~<n~ 



l-P q (n,r,n) = ^2(-l) b+1 q(2) 



6=1 



q- rb . 



Proof. The formula for \C m \ stated earlier in this section implies the exis- 
tence of A with A(e) = 0. Proposition 5.4 of |F6] states that for any A, 
if K r (l, A) > 0, then the largest part of A(e) is at least n — r. It follows 
that if A(e) = then K n_1 (i,A) = 0. By Theorem EE! and the fact that 
P q (a,n — l,n) > for < a < n — 1, it follows that if A(e) = then 
c a (A) = for < a < n — 1. Thus if A(e) = 0, only the a = n term in 
Theorem 14.31 can be non- vanishing, but this term is independent of A, since 
Cn }^ = 1 for all A. This implies the first part of the lemma since all terms 
in Theorem 14.31 are nonnegative. It also implies that s(r) = 1 — P q (n,r,n), 
and the equality in the second assertion follows from Lemma 14.21 □ 

Theorem [43] bounds the separation distance s(r) and determines its exact 
asymptotic behavior. 

Theorem 4.5. Suppose that GL(n,q) / GL(1,2). 
(1) If r < n, then s(r) = 1. If c > 0, then 

1 4 , , 2 



< s(n + c) < 



qc+l g2c+3 — v ' — qc+l 



(2) Lei c > be fixed. Then 

oo 

lim s(n + c) = 1 - TT (1 - g~( c+m )). 

m=l 

Proof. The first two sentences of the proof of Corollary 14.41 implies that if 
r < n, then s(r) = 1. To upper bound s(n + c), one checks that since 
q > 2, the sum in the second part of Corollary 14.41 is alternating with terms 
of decreasing magnitude. Thus the sum is upper bounded by its first term, 
["]g - ( n+c ), which is easily seen to be less than 2q~^ c+l \ since q > 2. For 
the lower bound, note that the first term in the second part of Corollary 14.41 

is at least g~( c+1 ) and that the second term — a n+ a e -i is a t least — 4q~( 2c+3 ^ 
since q > 2. This proves the first part of the theorem. 

To prove the second part, rewrite the expression for s(n + c) in part 2 of 
Corollary 14.41 as 

" (-l)fa(l - Vg^)(l/g - l/g") ■ ■ ■ (I/?"- 1 - l/q") 
2^ g(c+l)6(l-l/ g )...(l_l/^) 
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It is straightforward to see that as n — > oo this converges to 



oo 




(<H-l)&g(S)(l-l/ g )...(l-l/g*)' 



6=1 q 



An identity of Euler (Corollary 2.2 in [Anj ) states that 



OO 



DO 




G)(l-l/g)...(l-l/g6) 



II( 1 + ^- m ) 



m=0 



for \t\ < l,\q\ > 1. The result follows by applying this identity with i = 



For certain one dimensional random walks, namely stochastically mono- 
tone birth-death chains, Lagrange-Sylvester interpolation allows one to study 
separation distance knowing only the eigenvalues (and not the eigenvectors) 
of the Markov chain; see [Br] . |DSa] and the remarks following Proposition 
15.11 The purpose of this section is to give examples of higher dimensional 
state spaces (namely Irr(S n ) and Irr(GL(n,q))) where the methodology is 
useful. 

To begin we review the Lagrange-Sylvester interpolation approach to di- 
agonalizable matrices, and hence to reversible Markov chains. A textbook 
discussion in the matrix setting appears in [Gaj . and the paper |Br] uses the 
language of Markov chains. 

As usual, K is a Markov chain on a finite set X and is reversible with 
respect to a distribution tt. If n{x) > for all x, then letting A be a diagonal 
matrix whose (x,x) entry is vr(x), it follows that A 1 / 2 KA~ 1 / 2 is symmetric. 
Hence K is conjugate to a diagonal matrix D, whose entries d\, ■ ■ ■ ,d n are 
the eigenvalues of K. Thus if f,g are polynomials with fidj) = g{di) for 
i = l,--- ,nthen f(K)=g(K). 

Let Ai,--- , X m be the distinct eigenvalues of K (so m < \X\). Define 
g r (s) = s r and 



□ 



5. Lagrange-Sylvester Interpolation 



rn 





n 



i=i 



Since / r (Aj) = g r (\i) for i = 1, ■ ■■ ,m, it follows that f r (D) = g r (D). Thus 
f r (K) = g r (K) which gives that 
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As noted in [Br] , expanding this expresses K r in terms of I, K, ■ ■ ■ , K m 1 
as follows: 



i=l j^i a£c(m,i,rn—a) s€a 



a=l 



Here c(m, i,m — a) consists of the f^J subsets of size m — a from {j - : 1 < 
j <m,j ^ i}. 

For the next proposition it is useful to define the distance dist(x, y) be- 
tween x,y € X as the smallest r such that K r {x,y) > 0. For the special 
case of birth-death chains on the set {0, 1, • • • ,d}, Proposition 15.11 appears 
in [DF] and [Br] . 



Proposition 5.1. Let K be a reversible ergodic Markov on a finite set X. 
Let 1, Ai, • • • , \d be the distinct eigenvalues of K (so d + 1 < \X\). Suppose 
that x,y are elements of X with dist(x,y) = d. Then for all r > 0, 



K'(x,y) 



n 1 ~ Aj 



Proof. Since dist(x, y) = d, the Lagrange-Sylvester expansion of in terms 
of I, K, ■ ■ ■ ,K d gives that 

K*(x, y) = K d (x, y) I - A,)" 1 - £ A[(l - A,)" 1 J](A, - A,)" 1 

By Lemma l2.2( ergodicity of K implies that 7r(y) = lim r ^ OQ K r ~{x,y). Thus 
7r(y)=K d (x,y)n(l-A,)- 1 , 



which implies that 



i=l 



Ai — Aj 



□ 



Remarks: 

(1) Let K be a reversible Markov chain on a finite set X. As in Proposi- 
tion Em let 1, Ai, • • • , Xd be the distinct eigenvalues of K (so d+ 1 < 
|X|). From the expansion of K r in terms of I, X, • • • , .Jf* it follows 
that dist(x, y) < d for any states of the chain. Thus the hypothesis 
of Proposition 15.11 is an extremal case. 
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(2) Let K be a birth-death chain on the set {0, • • • ,d}, with transition 
probabilities 

a x = K(x, x — 1), x = 1, ■ ■ ■ ,d 

b x = K(x,x), x = 0, • • • , d 

c x = K(x, x + 1), x = 0, • • ■ ,d — 1. 

Suppose that a x > for < x < d and that c x > for < x < d. 
Such chains are reversible with respect to the stationary distribution 



7T(X) = ZH 



Cj-1 
CL; 



where Z is a normalizing constant. Supposing further that c x + 
a-x+i < 1 for < x < d (such chains are called monotone chains), 
then [DF] showed that the separation distance for the chain started 
at is equal to 

Applying Proposition 15,11 with x = 0,y = d one recovers the lovely 
result of Diaconis and Fill [DF| expressing the separation distance 
entirely in terms of the eigenvalues of K: 



i=i 



n 1 ~ Aj 



This fact was used in [DSa] to give a necessary and sufficient spectral 
condition for the existence of a separation cutoff for monotone birth 
death chains. 

Next we use Proposition 15 . 1 1 to give eigenvector- free proofs of the formulas 
for separation distance for the random walks on Irr(S n ) and Irr(GL(n,q)) 
analyzed in Sections [3] and HI It should be emphasized that as in the proofs 
of Sections [3] and [4J one still needs to know at what representation the 
separation distance is attained. 

Proof. (Fourth proof of part 1 of Theorem I3.5j) By Corollary 13.41 the sepa- 
ration distance is equal to 

{ ) *((i»)) ' 

where (l n ) is the partition corresponding to the sign representation. As was 
mentioned at the beginning of Section [3] and proved in F6], the Markov 
chain K has a description as a random walk on partitions in which one 
removes and adds a box at each step. From that description it is clear that 
the trivial representation (corresponding to the partition (n)) and the sign 
representation (l ra ) are distance n — 1 apart. By part 1 of Lemma 12.11 the 
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chain K has n distinct eigenvalues, namely ^ where 0<i<n — 2 or i = n. 
Thus Proposition 15.11 implies that 

n-2 / • x r ( 1 



*> - e n 



0<j<n-2 V '' 

n-2 

' J V -n- n 



E ; n 



8=0 v 7 i^i J 

0<j<n-2 

g^y^L n j_ 

z — ' \n J n — i - LJ - i — 7 

j=0 v 7 i^i J 

0<j<n-2 

eW'O-^)' 



□ 



A similar argument works for the general linear case. 

Proof. (Second proof of part 2 of Corollary 14. 4h By part 1 of Corollary 14.41 
the separation distance is equal to 

K r (l,A) 

where A is any representation satisfying A(e) = 0. By part 1 of Lemma 
12.11 the chain K has n + 1 distinct eigenvalues, namely q~ % for < i < n. 
By the first remark after Proposition 15.11 this implies that dist(l,A) < n. 
Proposition 5.4 of [F6| states that for any A, if K r (l,A) > then the 
largest part of A(e) is at least n — r; it follows that dist(l,A) > n. Thus 
dist(l,A) = n, and so Proposition 15.11 can be applied with x = l,y = A. 
Using the notation (l/q)k = (1 — l/<z) " " " (1 — l/<? fc )> one obtains that 



n 

"(r) = E^ r n ^ 



r-3 



q 1 - q ] 

1 = 1 3& 



l<j<n 

n , , v i-1 n 

E -ir Wgjn TT 1 TT 

8=1 v H ' 3 =1 y H j=i+l 

A _ ir (l/q) n (-ly-'q^ q^-Q 
^ q (l-q-i) (!/«&_! (l/g) n _ 



8=1 
n 

-ir 



E(-l) m ^- 



8=1 



9 



□ 
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